1. Field of the Invention
The present invention relates to an apparatus and method of augmented reality interaction.
2. Description of the Prior Art
Typical augmented reality (or ‘AR’) applications or systems receive live or recorded video images of a real environment, and then augment these video images with computer graphics in such a way that the computer graphics appear to move or be positioned in a manner consistent with the movement or position of the real environment in the video images.
The effect is to insert the computer graphics (or ‘virtual objects’) into the real environment in a consistent and believable way. Preferably this is done in real time; that is to say, the generation and augmentation of the video is performed at normal video frame rates. A good example of this can be seen in the game ‘Eye Pet’® for the Sony® PlayStation 3® or ‘PS3’®.
Most AR applications or systems achieve this by making use of a real object in the environment whose appearance and dimensions are known and in advance encoding the appearance and dimensions of this object as a reference model in a computer. By comparing the scale and orientation of this object as found in the video images with the reference model, it is possible for the computer to calculate the corresponding scale and orientation that should be applied to virtual objects used to augment the image.
To improve the reliability of this process in adverse lighting conditions, or where the video camera in use has a low resolution, often the real object used is a so-called fiduciary marker 800, an example of which is shown in FIG. 1. Such markers typically have a high contrast border and patterning to improve robustness to lighting, and the pattern is typically asymmetric to help resolve the orientation of the marker.
Often, in order to maintain the illusion of the augmented reality environment, a virtual object added to the video image is positioned over the fiduciary marker in the video image so that it is not visible in the augmented image. This improves the aesthetic of the resulting augmented images, but means that if a user wishes to interact with the virtual object (or example stroking a virtual pet), there is a significant risk that they will occlude the fiduciary marker as they will be gesturing within the real space close to it. Depending on the extent to which the marker is occluded in this manner, the application or system may lose the ability to identify the marker and/or accurately determine its scale or orientation, making realistic augmentation of the video images difficult and resulting in the graphical objects moving unnaturally with respect to the background video, spoiling the AR effect. As this tends to occur just as the user is attempting to interact with the object, the effect can be very frustrating, particularly for younger users.
Consequently there is scope to improve means by which users can interact with virtual objects that overlie fiduciary markers.